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Abstract 

In large residential treatment centers for adolescent youth, program administrators and clinical staff rely on 
the information imparted to them by direct care staff to make appropriate decisions regarding administrative and 
clinical support functions so that the residents in care can receive the best treatment possible. This study was 
designed to increase the efficiency of program status reporting by direct care residential staff in a large residential 
setting serving an average daily census of 95 youth residing in four distinct residential programs: including programs 
for sexually abusive adolescents, sexually reactive adolescents, adolescents with anger management problems, and a 
latency age children’s program. Staff were trained to implement a 13 item checklist specifying key information such 
as the number of critical incidents, physical holds, family contacts, completion of scheduled unit activities and 
completion of required documentation at the end of each residential shift. Direct care staff were trained to 
implement the status report protocol using a staff management package including antecedent procedures consisting 
of instructions, role-playing and feedback on the use of the reporting protocol and consequent procedures comprised 
of written and verbal feedback provided by a supervisor to staff on a weekly basis. The feedback given to direct care 
staff consisted of the percentage of items they reported correctly from the 13-item status reporting protocol. The staff 
management intervention package was assessed employing a multiple baseline design across the four programs. 
Reliability data were collected on an average of 58% of the reports given by staff during baseline and intervention. 
The data indicated that the staff management package was successful in increasing the efficiency of program status 
reporting by direct care staff. 
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The design and successful operation of social technologies such as residential programs for children and 
youth are guided in part by legal policy, judicial decisions, governmental regulations, and standards set 
forth by accrediting bodies such as the Joint Commission on Accreditation of Healthcare Organizations 
(JCAHO), the Council on Accreditation (COA) and Commission on Accreditation of Rehabilitation 
Facilities (CARF). It is noted that the “application of legal policy may affect any aspect of human service, 
in intervention design it generally entails at least the specification of guidelines involving client rights and 
practitioner, agency, program, or institutional responsibilities.” (Thomas, 1984, p. 128). For example, the 
Massachusetts Office of Child Care Services (OCCS) specifies clear guidelines for the documentation and 
reporting of restraints and critical incidents involving clients residing in residential programs for children 
and youth. 

Group care facilities write agency policy and procedures to bring their youth care practices into 
compliance with licensing regulations. In turn, these policies and procedures are taught to staff so they 
can implement them with the residents in their care and insure that the organization adheres to appropriate 
legal policy set forth by licensing and accrediting agencies. However, even if a residential program 
adheres to licensing requirements, this does not imply that the program is providing quality care, 
particularly if licensure is not based on naturalistic observations of staff and client behaviors (Repp & 
Barton, 1980). Even organizations 
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that are successfully certified via procedures such as the federal Medicaid program survey process cannot 
be 

successfully discriminated from agencies that are not certified, particularly in regard to the treatment of 
clients. In addition, a reactivity analysis of one such survey indicated that direct care staff performed 
differently during the survey by increasing interactions with clients and decreasing non-work behavior. It 
is also been noted that when provider agencies are polled regarding the consistency, accuracy, and 
objectivity with which survey teams determine agency compliance with accreditation and certification 
guidelines, they report considerable variability in the assessment of standards by accrediting agencies 
(Reid, Parsons, Green, & Schepis, 1991). 

One possible explanation for the inability of policies and legal guidelines to consistently occasion 
appropriate staff behavior is that they function as antecedent management techniques in the form of 
instructions to staff. Memos, public posting of responsibilities, scheduling of staff assignments, 
specification of policies, goal setting, general explanations regarding what a job entails and specific 
instructions detailing what a task involves, where it is to be done, when to do it and with whom are all 
examples of instructional techniques often employed to occasion desired staff behaviors (Andracik, 
McNamara, and Abbott, 1978). In spite of the fact that instructional methods are among the most 
commonly employed techniques for managing staff behaviors (Madle, 1982), little behavior change 
results from these methods (Iwata, Bailey, Brown, Foshee, and Alpem, 1976). Generally, the use of 
instructional techniques alone produces no change in staff performance (Quillitch, 1975), minimal change 
or short-lived change (Pommer and Streedbeck, 1974). 

It is not surprising that job performance is not greatly affected by the exclusive use of traditional 
instructional methods. Instructional methods for motivating staff performance may be considered as un- 
reinforced instructions to staff and as such would not be expected to exert a powerful effect over their 
performance (Ayllon and Azrin, 1968). Instructions will control staff behavior if they are followed-up and 
reinforced through other management techniques like performance based feedback (Kurtz, Lutzker, Cuvo, 
Eddleman, Lutzker, Meagson, and Gulley, 1982). 

The value of instructional procedures is that they help to set the occasion for the effectiveness of 
other staff management procedures like performance based feedback (Seys and Duker, 1978). In addition, 
instructional techniques that specify desired work behaviors make it easier for managers to monitor the 
occurrence of these behaviors (Risley and Favell, 1979). Even though instructional techniques alone have 
shown little positive impact, they are a desirable and necessary adjunct to other more effective staff 
management procedures (Connellan, 1978). 

The previous discussion underscores the necessity for supervisors to monitor, follow-up and 
reinforce those behaviors that have been taught to staff through antecedent performance procedures such 
as modeling, role-playing, prompting, verbal instructions, memos, goal setting and publication of 
organizational policies. Unless follow-up occurs, it is unlikely that the time and energy invested in the use 
of these various procedures will result in desired changes in staff performance. 

Effective models of staff management consist of multifaceted management procedures in which a 
large number of different techniques are combined into a maximally powerful intervention package using 
resources indigenous to the organization (Ivancic, Reid, Iwata, Faw, & Page, 1981). Previous research in 
residential settings has established the efficacy of staff management packages for imparting such diverse 
staff skills as nutritional practices in community-based group homes (Kneringer, & Page, 1 999), student 
participation in functional activities (Dyer, Schwartz, & Luce, 1984), and direct care staff teaching 
behaviors (Page, Iwata, & Reid, 1982). However, few studies have demonstrated the utility of staff 
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management packages for improving the implementation of agency policy and procedures by direct care 
staff. The focus of this study was to increase the efficiency of program status reporting by direct care staff 
in accordance with agency policy and OCCS legal licensing requirements utilizing a staff management 
package including instructions, modeling, behavioral rehearsal, staff monitoring and supervisor feedback. 


METHOD 


Participants 

Twenty-five direct care staff, in most instances senior counselors, who worked in seven 
separate units of a children’s residential treatment facility served as subjects. These individuals 
typically worked three days during the week from 2:00pm - 10pm and one weekend day from 
8am - 10pm. As the senior lead-on counselor they were charged with the responsibility for 
giving the unit status report covering events and activities that occurred during their shift, i.e., 

10:00 P.M. Verbal status reports by the senior lead-on counselor were made to Program Directors via 
telephonic voice mail messages at lOp.m each evening at the end of their respective shifts. 

Setting 


The facility is a ninety-nine bed residential child care facility located in a semi-urban 
setting in close proximity to a major city in western Massachusetts. The client population 
consisted of children and youth between the ages of 6 and 17 involved in four distinct clinical 
programs housed in five cottage settings: latency age boys, pre-adolescent and young adolescent 
girls with histories of chronic complex trauma, sexually reactive latency age boys, male 
adolescent sexual offenders, and adolescent males with anger management issues. All clients 
manifested significant and varying degrees of the following symptoms: behavioral, psychological, 
interpersonal, psychiatric and neuropsychologic and all exhibit DSM-IV-TR diagnoses. All 
children attended the center’s therapeutic school located on the campus adjacent to the residences, 
with the program offering a range of special educational services as well as behavior 
management programs designed to interface with residential treatment goals and objectives. A 
variety of psycho-educational, psychosocial, psychotherapeutic, psychiatric, and 
psychopharmalcologic treatments were offered depending upon the child’s unique situation and 
need. Mean length of stay is approximately 1 6 months. 

Instrumentation 

A 13-item, forced choice (Yes/No) checklist, the Status Report Evaluation Checklist 
(SREC), was used to capture essential client information needed to insure consistent and 
effective communication between all shifts. The checklist captures information in relation to 
first and second shifts: global unit status (excellent, good, fair, poor), number of corrective 
physical interventions (CPI), critical incidents, (e.g., fights, injuries, AWOL’s, etc.), family 
contacts (visits and telephone), scheduled Unit activities (trips, shopping), and reference of 
documentation being completed; i.e., incident reports (see appendix A). The SREC was designed 
to function as a tool to obtain comparative data across staff and programs, serve as a self- 
monitoring device for staff, and as written record of feedback on their performance 
(McClannahan, McGee, MacDuff, & Krantz, 1990). 
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Measurement 

On a daily basis Program Directors listened to telephonic status reports via agency voice 
mail recording made by direct care staff during both baseline and treatment periods and then 
rated the reporter’s effort utilizing the SREC. Results documented the percentage of required 
information given by staff. An independent rater also listened to a sample (58%) of both the 
baseline and treatment reports and compiled reliability information. Inter-rater reliability data for 
the baseline period was 88.77% based on a random sample of 77 reports, while the inter-rater 
reliability measures for the treatment period increased to 93.36% based on a sample of 121 
reports. The overall inter-rater reliability for both baseline and intervention measures averaged 
91.07%. 


Procedure 

A multiple baseline design across settings was used to evaluate the effects of training 
and feedback. During both baseline and treatment the subjects were unaware of the dependent 
variables being evaluated. 

Baseline: During baseline, the subjects followed typical agency procedure for making 
program status reports (see appendix B for a copy of the agency policy and protocol). No written 
or verbal feedback was given to staff regarding the completeness or accuracy of their status 
reports. Baseline measures were begun simultaneously across seven distinct units: units 4, 2A, 2, 

3, 5, 6, and 6A. 

Intervention: A performance management training package was employed to re-train direct care 
staff on how to make program status reports utilizing the SREC. The performance management package 
consisted of antecedent techniques including instructions regarding the status report policy and procedure, 
introduction of the SREC, modeled use of the SREC, as well as behavioral rehearsal and feedback on the 
use of the SREC. The trainer informed staff they would also receive weekly written and verbal feedback 
on their performance of the use of the SREC. In addition, staff were given laminated copies of the SREC 
and instructed to use them to monitor their performance during the reporting of program status reports. 
Program Directors monitored their respective staff giving status reports on a daily basis and provided 
weekly written feedback to them in terms of the number of correctly reported items on the SREC as well 
as verbal qualitative feedback on their performance. The staff management package was implemented 
across programs and units in a staggered fashion in intervals of one week utilizing a multiple baseline 
design. 


RESULTS 

Figure 1 shows the multiple baseline across programs for all seven units involved in the study. 

All staff behaviors across the four programs improved as a result of the application of the staff 
management package. The percentage of items reported correctly on the program status report increased 
from a mean baseline of 45.59% to 97.38% during intervention. Measures for each of the units were as 
follows: U4 increased from 44.43% to 96.5%, U2A increased from 28% to 100%, U2 increased from 23% 
to 95.6%, U3 increased from 60.95% to 98.12%, U5 increased from 64% to 94.14%, U6 increased from 
49.46% to 99.62%, and U6A increased from 45.49% to 97.38%. 
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During baseline status reports were missing on 6.8% of the total measures taken. A total of 7.82% 
of the measures during intervention were missing due to problems with the phone system not working 
properly, untrained staff filling in at the last minute for scheduled staff who called in sick and unexpected 
staff call outs from work. 
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DAYS 


DISCUSSION 

A staff management package consisting of instructions, modeling, behavioral rehearsal, staff 
monitoring and performance feedback was successful in increasing the percentage of items that direct 
care staff reported via the agency status reporting system in accordance with agency policy and 
procedure. Even though staff had been previously trained on how to make program status reports, this 
training was insufficient to impart the necessary job skills. It was only when the staff management 
package was implemented did staff attain high levels of agency compliance with the status reporting 
protocol. This study underscores the finding that training alone, especially on agency policy and 
procedures has limited value in imparting the skills that a policy and procedure prescribes (Ziamik, & 
Bernstein, 1982). 

The results of this study also call into question the value of outside surveys and reviews as an 
effective mechanism for insuring program quality. It can be argued that the review process can be 
conceptualized as a highly organized set of antecedent procedures and as such would not exert long-term 
control over the staff behaviors it is intended to prescribe and reinforce. Program administrators should 
not 
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rely on the feedback from reviewers as an effective mechanism for insuring that staff will carry out 
recommendations made through the review process. Supervisors need to conceptualize program 
implementation not only as training and feedback on what needs to be done, but also follow-up and 
reinforcement on what is actually done (Parsons, & Reid, 1994). 

One of the limitations of this study is that it is hard to know what specific elements of the staff 
management package were responsible for changes in staff behavior. For example, it is not clear to what 
extent staff self-monitoring via the use of the SREC played in the overall results versus the provision of 
performance feedback (Richman, Riordan, Reiss, Pyles, & Bailey, 1988). It is also not easy to determine 
whether the daily feedback to Program Directors on their direct care staffs behavior via the SREC 
affected the quality of performance feedback they provided to staff (Page, Iwata, & Reid, 1982). 

It is interesting to note that two off campus group home programs that were not included in this 
study spontaneously began using the SREC protocol during the study with no direction or training from 
supervisors. Anecdotal data indicated that staff began implementing a higher level of program status 
reporting once the SREC protocol and staff management package were implemented in other programs 
within the agency. Further study should be conducted to see how direct care staff implementation of 
policy and procedures can be affected when not directly trained and reinforced. 
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